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ABSTRACT * • 

A patient education evaluation project and how it 
applies or aight be useful to other hospital settings is discussed in 
this paper* The following topics are included: United Hospitals {St* 
Paulir Minnesota) ; situation and approach to patient education; 
description .of the patient education evalua^tion project, including 
$esign> criteria, and eleaents of the systea (prograa objectives, 
aeasures, perforaance expectancies, weights {ranking of objectives], 
client descriptors, and the aanagement report) ; and the systea's use - 
in other hospital settings. (TA) , , 
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This afternoon, I would like to discuss wxth you our pataent education 



XniS aLUerUUULl, J. w<juau ^^r^^ y^j^^y^^^^ y ^ — 

^ ^ evaluation project and^ how it applies or might be usefu?. to other hospital 
settings. To set a framework for this presentation, let me first briefly 
describe our hospital situation and our approach to patient education. 



Overview 

United Hospitals, which is the resulting corporation of a merger of Miller 
and St. Luke's Hospitals, has 720 beds and is located in downtown St. Paul, 
Minnesota. The hospital is encouraging health protection and promotion 
through a variety of approaches that include patient education, health promo- 
tion and safety services for its employees, and a wide variety lof health educa- 
tion programs operated outside the hospital for community re'si^ents. Health 
education at United Hospitals is the responsibility of a department organized 
in 1970 to guide the development and growth of the hospital's educational ser- 
vices for patients, employees, paramedical students on affiliation at United, 
and community members, as well as to serve in a liaiso^n capacity with the med- 
ical education department. Its functions also includeMirection of the. staff 
and patient libraries, provision of medical photography services, and super- 
vision of all audiovisual resources. 

Since 1969, United Hospitals has become increasingly more involved in and 
committed to providing health education services for its patients. Starting 
with health education services for persons with diabetes, our hospital staff 
now provides formalized health education services as a routine component of 
care to inpatients and outpatients with diabetes , several forms of heart 
disease, and chronic obstructive pulmonary disease. In addition, a pilot 
for a hospital-wide program in self-administration of medications for ail 
appropriate patients will begin in October 1976. 

Description of Our Approach to Patient Education 

We believe that patient education can be approached as a specific treatment 
modality. Our concept of patient education is that it consists of planned 
health educational ^experiences for a patient that are designed by his phy- 
sician, professional health workers, the patient's family members, and the 
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.patient himself to meet the patient's specific learning needs, interests, and 
capabilities and are offered as an integral part of the patient's total 
health care. 

Some of the more^ important characteristics of oar patient education programs 
are that: 

' 1. They are multidisciplinary in nature. 

2. They use team planning and teaching. ; ' - 

3. They use as teachers the professional health workers (such as nurses 
/pharmacists^ dietitians, etc.) who are already caring for the 
patient other health needs. 

4. Standard methods are used for: 

a. initiating the patient and family education. (Similar to a 
physician's ordering any other service.) 

b. Determining the kind of education a patient and his family 
need. 

c* Setting reasonable behavioral objectives to be attained 

through education. ^ 

d. Delivering the planned educational activities. 

e. Evaluating the effectiveness of the education. 

f . Making available to the patient and his family further 
guidance in self care after discharge from the hospital, 
if this is needed. 

Patient Education Evaluation Project 

/Evaluation of our patient education efforts has been an ongoing challenge. 
I As a service-oriented rather than a research-oriented facility, the .emphasis 
I of our evaluation efforts has been directed toward: 

i - 

I 1* The development of an evaluation system (ongoing mechanism), which 

I can be used to monitor our progress, pinpoint where changes are 

I . needed, and suggest ways to relate to the community how we are 
using their money. 

2. The development of a usable management tool which can tell us the 
results of services provided (impact and quantification.) 

A year ago. United was one of six-member agencies of the Human Resources 
Planning Council of the St. Paul area to be chosen to participate in a 
Program Effectiveness Project headed by Walker and Associatesf a local con- 
sultant firm. ,The intent of this project is to develop an evaluation system 
that will measure both the effectiveness and the efficiency of the program 
to be evaluated. 

T " ' — " ' ' 

Bob Walker and Associates, 23 East Grant St., Minneapolis, Minnesota 55403 
(612-332-6377). The inclusion of the name of this company and itsi methodol- 
ogy does not constitute endorsement by the U.S. Department of Health, Educa- 
tion, and Welfare or any of its agencies or by the American Hospital Associ- 
ation. 
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\jtilizing the elements. of a formative program evaluation system as\defined by 
Wdiker, we have been able to clearly identify what we are striving :to accom- 
plish, the services we are providing, who the program serves, and shortly the. 
results of our efforts on a continuing basis. We are using this information 
to help market our programs, gain further community (and hospital)., support, 
and continually improve program results. 

The basic assumptions upon which the system is designed aire that: 

• * t 

1. There is a cause and effect relationship between services^^ provided 
and results obtained. 

2. Knowledge o2 results ipproves program performance. In others-words, 
if -feedback is given to^staff on results of their efforts, this,, ^ 
knowledge helps the providers do a better job. Holding staff account- 
able for results tends to improve productivity. ^ 

From Walker's perspective, program evaluation is^a systematic procedure for 
regularly determining the results achieved by persons following the provision 
of^services and determining the efficiency with which those results are ob- 
tained. 

The criteria for a program evaluation system include the following: 

1. Program evaluation measures what happens to all clients. 
. 2. Measures are applied outside of the facility. 

3. Reports on results are ^produced at regular intervals (monthly, 
quarterly) . 

4. Reports tell you whether or not your performance was acceptable. 
The 8 basic elements of our program evaluation system are 

1. A statement of purpose or mission 

Definition: The purpose or mission statement of an organization 
describes, in general ways, what it strives to accomplish, the 
services provided and who it serves. It should be sufficiently 
broad to cover all the organization's programs but specific enough 
to distinguish it from other organizations in the cCifflnunity. 
Our mission statement for patient education: to provide edu- 
cational services under medical supervision to the chronically 
disabled which will reduce premature patient mortality and enable 
such patients to maximize capacities for independent living. 

2. Program Goals 

Definition: Program r^oal statements are more specific than the 
organizational purpor,e and describe for each of the programs the 
kinds of services provided, who is served, and the kinds of results 
to be achieved. An organization with three programs would have 
program goal statements for each of the three programs. 
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Criteria, for Program Goals: 

a. Mii3t state more specifically who' is sexrved by program, 
what- services ai:e provided, and ;what the program is de- 
signed to do. 

b. Program goal statement must be specific enough to allow 
one -to determine the program objectives^ 

. c. Must be achievable. Goal should be able to be accomplished 
by the services provided\ 

The program goal statement for our cardiac patient education program: 
To provide an individualized program of education, skill development, 
and counseling seirvices to cardiac patients and their families 
which xrtll minimize premature mortality, reduce inappropriate hos- 
pitalizations, and facilitate their ability to live independently. 

Program Objectives ^^^^^^^ ^ 

Definitions Objectives are the very specific statements of what is 
to be achieved by a program. If the specific objectives are achieved, 
then the program will have accomplished its goal. Objectives are the 
statements from which measures are derived and need to be stated 
in terms of the results to be achieved. All programs should have 
objectives covering both effectiveness and efficiency. 

Criteria for Objectives: 

a. Objectives^ must be measurable. 

b. The sum of all objectives equals accomplishment of the goal. 

c. All, programs should have objectives^covering effectiveness 
and efficiency. \ 

.Objectivesfs for Our Cardiac Patient Education Program: 

a. Maximize the number of patients who do not require an - " 
in-home service. 

b. Minimize the number of patients requiring institutional care. 

c. Maximize personal independence. 

d. Reduce social isolation. 

e. Maximize the number of patients who follow critical recom- 
mendations in the home treatment plan.l 

f . Obtain employment or return to work. 

g. Reduce need for inappropriate rehospitalization. 

h. Maximize use of the educational program. 
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Measures 

Definition: Measures are the statements which indicate how the j 
achievement of objectives will' be. determined. Measures should be 
reliable and applied following the provision of services. The / 
measures must clearly indicate to whom they are to be applied and 
when they will be done. / 

CriteriaTror Measures:* * - . , I \ 

a. Must measure whether the objectives are achievaljle (valid - 
reliable) . ♦ 

b. Must supply accurate information. ; 

c. Should measure a final result occurring outside of the 
- facility. ; 

Measures -for Our Cardiac Patient Education Program: 1 

* 

a. Percentage of patients who do not require an iu-home service^ 
in the past 2 weeks prior to follow-up. 

b. 'Percentage of patients not in an institution. 

c. Percentage of patients who independently perform tKose crit- 
^ leal living skills learned or encouraged by the educational 

p rogram (See sp ecial instrument) . _ | _ 

d. . Percentage of patients who left place olE residence^ for ~ 

recreational or social purposes In the past month prior 
to follow-up. 

e. Percentage of p'atients who follow critical recommendations 
in the home treatment plan (See special instrument) i 

f. Percentage of patients who have returned to work or obtained 
new employment. . 

g. Percentage of patients who are not rehospitalized; for the 
condition for which they received educational services.- 

h. Percentage of all primary and secondary diagnosed patients 
who are ordered to receive an educational program. 

Performance Expectancies 

Definition: Expectancies are statements of the degree to which each 
objective is achieved. They are the criteria against which actual 
performance of each objective can be compared. 

Criterion For Expectancies: 

a. Expectancies must be set at a level which encourages improve- 
ment. 

' Factors to Consider in Developing Expectancies: 



a. Past performance. 
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b# . ^Thdm yoir serve, 
c. Environment (control) . 
d* Quality of your service, 
e. Quantity of your service. 

f ♦ Note: Expectancies should not be set without consultation 
of those who will do the work; theys should be negotiated 
with staff. 

Expectancies are set at three levels (they can be changed and must b( 
to remain reasonable ) : 

a. Goal level - if achieved, this would be satisfactory 
performance. 

b. xtinimal - below this, if program cannot be improv.ed, it 
shbuld be eliminated. 

c. Optimal - this is the upper limit, if everything goes right 
this is what you can ac*hieve, (not necessarily the maximum 
possible) . 

Expectancies should be reasonable for the program's clients. 

a. Expectancies help to measure relative, not absolute, per- 
formances. 

^ ^ - •Setting..expectancieS"raakes--the-system"explici-tri30~everyone 
is playing under the same rules. 

Sample expectancies: Heart Program ^ 

Minimal Goal Optimal 

Percentage of patients who did not 
require an in-home service in the- 

past 2 weeks prior to follow-up. 70% 80% 90% 

Weights 

Definition: The relative^ importance of objectives to be achieved by 
each program need to be indicated. If program has more than one 
objective afid measure and they are not all of equal importance, they 
should be' either rank-ordered or assigned relative weights. 

How weights are set: 

a. Weights are set dn terms of percentages. 

b. . All of the objectives for each program goal are assigned 

weights. 

c. Total = 100%. 



Weights enable data to^ be combined in a single score of overall 
program performance. 

With patient education, look at the balance between acquiring be- 
havior and utilizing belfavior. Weights can help change staff and 
patient behavior. The highest priority is the objective with the 
heaviest weight. 

7# Client Descriptors 

The evaluation system should describe the persons -served in such a 
way that it facilitates interpretation of the results obtained. The 
descriptors should indicate the severity of the problems to be re- 
solved by the services and indicate the major barriers%to their ^ 
achievement. 

Criteria for Client Descriptors: • 

Client descriptors should be defined in such a way that you can judge 
the adequacy of your results. 

Examples From Our Cardiac Program: 

a. Percent of patients over 65 years of age. 

b. Percent of patients who have diminished mental capa^^ity. 



8. Manaj^ement Report 

In order to accomplish the purposes of a program evaluation system, 
the performance data obtained must be regularly distributed to 
persons both within and outside- the organization. 

These reports should indicate the actual performance achieved on 
all the objectives, the expected levels of achievement, and a suiranary 
of the major descriptors of the persons on whom the performance 
information is based. 

Index scores are numerical figures which tell you the degree 

to which each objective is best achieved. ( 

Indexing system : 50 = minimal performance 
100 = goal p€».rformance 
150 = optlmax performance 

Combining this program evaluation system with some of our current evaluation 
methods (which include chart audit case studies, pre- and- post education 
assessments, and verbal feedback jfrom patients, family members, physi,cians 
and staff ) and Incorporating some excellent ideas and methods suggested by 
Lawrence W. Green, Dr.PH, of Johns Hopkins University to handle correctly three 




major variables which affect the attainment of patient education goals - 
namely, (1) the medical management plan, (2) the patient's physiological and 
psychological state, and (3), the patient's behavior - should give us a clear, 
useful picture of the effects and efficiency of our patient education efforts. 



Use in Other Hospital Settings 

Now let us examine for a moment how our approach to evalu?::ion applies or might 
be useful to other hospital settings. As- I discussed earlier in this paper, as 
a~service-oriented rather than a research-oriented facility, the emphasis of 
our evaluation efforts has been directed |:oward: 

1. The development of an evaluation system (ongoing mechanism) which , can 
be, used tb monitor our program, pinpoint where changes are neejied, 
and suggest ways to relate to the community how we are using cheir 
money. , ' 



2. The development of a usable management tool whica can tell lis the 
results of services provided (impact data), and which quantifies it. 

« 

Utilizing the elements of a formative program evaluation system, a hospital 
should be able to identify clearly what they are striving to accomplish, the 
services they are providing, whom the program serves, and the results of their 
,efforts on, a continuing basis. This information can be used to help market 
their programs, gain community and hospital support, and continually improve 
program results. , 

L strongly believe that most service organizations would greatly benefit from 
the utilization of a program evaluation system such as the one sugg^f^ed by 
Walker . 

As a part of the program effectiveness project, the six agencies involved 
(Health Resources Administration, St. Paul Pxehabilitation Center, Merrick 
Coinmunity Center, Migrants in Action, St. Paul Association for Retarded 
Children) completed a questionnaire for Walker. One of the questions asked 
was: 

IThat do you perceive as being the primary b enefit to your agency of 
program evaluation? 

I list their comments to summarize the versafity, usefulness, and necessity 
of evaluation of patient education programs. 

Comments : 



"It provides a useful picture of where our program is and wher^ 
it is going. It is being used as a planning tool. The data pro- 
vided could possibly be used to increase needed services. 



It has helped us to clarify our goals and_list them by priority - 
valuable information in making future budget requests. ^ 

6an be used as a cost/savings device. We can help people get^the 
services, they really need; yet riot use services which are unnecessary 

This system gives us clear an4 concise data that can be interpreted 
and^used to continually upgrade the results of our services." 
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